{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qR8Am0lBtRAx"
   },
   "source": [
    "# Ungraded Lab: Effect of Compacted Images in Training\n",
    "\n",
    "In this notebook, you will see how reducing the target size of the image datasets will affect the architecture and performance of your model. This is a useful technique in case you need to speed up your training or save compute resources. Let's begin!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qxY7KvGQ2Qdr"
   },
   "source": [
    "## Import the Libraries\n",
    "\n",
    "Start with importing all the libraries you will need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import random\n",
    "import numpy as np\n",
    "from io import BytesIO\n",
    "\n",
    "# Plotting and dealing with images\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "# Interactive widgets\n",
    "from ipywidgets import widgets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o-qUPyfO7Qr8"
   },
   "source": [
    "## Load the Dataset\n",
    "\n",
    "Define the directories containing the images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NR_M9nWN-K8B"
   },
   "outputs": [],
   "source": [
    "TRAIN_DIR = 'horse-or-human'\n",
    "VAL_DIR = 'validation-horse-or-human'\n",
    "\n",
    "# Directory with training horse pictures\n",
    "train_horse_dir = os.path.join(TRAIN_DIR, 'horses')\n",
    "\n",
    "# Directory with training human pictures\n",
    "train_human_dir = os.path.join(TRAIN_DIR, 'humans')\n",
    "\n",
    "# Directory with validation horse pictures\n",
    "validation_horse_dir = os.path.join(VAL_DIR, 'horses')\n",
    "\n",
    "# Directory with validation human pictures\n",
    "validation_human_dir = os.path.join(VAL_DIR, 'humans')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "z1wrZCxTPw4m"
   },
   "source": [
    "You can check that the directories are not empty and that the train set has more images than the validation set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "psLXEVanK6eW"
   },
   "outputs": [],
   "source": [
    "train_horse_names = os.listdir(train_horse_dir)\n",
    "print(f'TRAIN SET HORSES: {train_horse_names[:10]}')\n",
    "\n",
    "train_human_names = os.listdir(train_human_dir)\n",
    "print(f'TRAIN SET HUMANS: {train_human_names[:10]}')\n",
    "\n",
    "validation_horse_hames = os.listdir(validation_horse_dir)\n",
    "print(f'VAL SET HORSES: {validation_horse_hames[:10]}')\n",
    "\n",
    "validation_human_names = os.listdir(validation_human_dir)\n",
    "print(f'VAL SET HUMANS: {validation_human_names[:10]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZTpdVrBg2LZC"
   },
   "outputs": [],
   "source": [
    "print(f'total training horse images: {len(os.listdir(train_horse_dir))}')\n",
    "print(f'total training human images: {len(os.listdir(train_human_dir))}')\n",
    "print(f'total validation horse images: {len(os.listdir(validation_horse_dir))}')\n",
    "print(f'total validation human images: {len(os.listdir(validation_human_dir))}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5oqBkNBJmtUv"
   },
   "source": [
    "## Build the Model\n",
    "\n",
    "The model will follow the same architecture as before but they key difference is in the `input_shape` parameter of the first `Conv2D` layer. Since you will be compacting the images later when you define the `tf.data` datasets, you need to specify the expected image size here. Instead of 300x300 as in the previous two labs, you will specify a smaller 150x150 array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PixZ2s5QbYQ3"
   },
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "    # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
    "    # This is the first convolution\n",
    "    tf.keras.Input(shape=(150, 150, 3)),\n",
    "    tf.keras.layers.Conv2D(16, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2, 2),\n",
    "    # The second convolution\n",
    "    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The third convolution\n",
    "    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The fourth convolution (You can comment the 4th and 5th conv layers later to see how it affects the results)\n",
    "    # tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    # tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # The fifth convolution\n",
    "    # tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
    "    # tf.keras.layers.MaxPooling2D(2,2),\n",
    "    # Flatten the results to feed into a DNN\n",
    "    tf.keras.layers.Flatten(),\n",
    "    # 512 neuron hidden layer\n",
    "    tf.keras.layers.Dense(512, activation='relu'),\n",
    "    # Only 1 output neuron. It will contain a value from 0 to 1 where 0 is for 'horses' and 1 for 'humans'\n",
    "    tf.keras.layers.Dense(1, activation='sigmoid')\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s9EaFDP5srBa"
   },
   "source": [
    "You can see the difference from previous models when you print the `model.summary()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7ZKj8392nbgP"
   },
   "outputs": [],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PEkKSpZlvJXA"
   },
   "source": [
    "You will use the same settings for training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8DHWhFP_uhq3"
   },
   "outputs": [],
   "source": [
    "model.compile(loss='binary_crossentropy',\n",
    "              optimizer=tf.keras.optimizers.RMSprop(learning_rate=0.001),\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Sn9m9D3UimHM"
   },
   "source": [
    "### Data Preprocessing\n",
    "\n",
    "Now you will instantiate the datasets. As mentioned before, you will be compacting the image by specifying the `target_size` parameter. See the simple change below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ClebU9NJg99G"
   },
   "outputs": [],
   "source": [
    "# Instantiate the training dataset\n",
    "train_dataset = tf.keras.utils.image_dataset_from_directory(\n",
    "    './horse-or-human/',\n",
    "    image_size=(150, 150),\n",
    "    batch_size=32,\n",
    "    label_mode='binary'\n",
    "    )\n",
    "\n",
    "# Instantiate the validation dataset\n",
    "validation_dataset = tf.keras.utils.image_dataset_from_directory(\n",
    "    './validation-horse-or-human/',\n",
    "    image_size=(150, 150),\n",
    "    batch_size=32,\n",
    "    label_mode='binary'\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, you will rescale the images and set them up for training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2kS1a0dXLgY8"
   },
   "outputs": [],
   "source": [
    "# Define the rescaling layer\n",
    "rescale_layer = tf.keras.layers.Rescaling(1./255)\n",
    "\n",
    "# Rescale both datasets\n",
    "train_dataset_scaled = train_dataset.map(lambda image, label: (rescale_layer(image), label))\n",
    "validation_dataset_scaled = validation_dataset.map(lambda image, label: (rescale_layer(image), label))\n",
    "\n",
    "SHUFFLE_BUFFER_SIZE = 1000\n",
    "PREFETCH_BUFFER_SIZE = tf.data.AUTOTUNE\n",
    "\n",
    "# Configure the training set\n",
    "train_dataset_final = (train_dataset_scaled\n",
    "                       .cache()\n",
    "                       .shuffle(SHUFFLE_BUFFER_SIZE)\n",
    "                       .prefetch(PREFETCH_BUFFER_SIZE)\n",
    "                      )\n",
    "\n",
    "# Configure the validation dataset\n",
    "validation_dataset_final = (validation_dataset_scaled\n",
    "                            .cache()\n",
    "                            .prefetch(PREFETCH_BUFFER_SIZE)\n",
    "                           )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mu3Jdwkjwax4"
   },
   "source": [
    "### Training\n",
    "\n",
    "Now you're ready to train and see the results. Note your observations about how fast the model trains and the accuracies you're getting in the train and validation sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Fb1_lgobv81m",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "history = model.fit(\n",
    "      train_dataset_final,\n",
    "      epochs=15,\n",
    "      validation_data = validation_dataset_final,\n",
    "      verbose=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the training and validation accuracies:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the training and validation accuracies for each epoch\n",
    "\n",
    "acc = history.history['accuracy']\n",
    "val_acc = history.history['val_accuracy']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(len(acc))\n",
    "\n",
    "plt.plot(epochs, acc, 'r', label='Training accuracy')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend(loc=0)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o6vSHzPR2ghH"
   },
   "source": [
    "### Model Prediction\n",
    "\n",
    "As usual, it is also good practice to try running your model over some handpicked images. See if you got better, worse, or the same performance as the previous lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the widget and take care of the display\n",
    "uploader = widgets.FileUpload(accept=\"image/*\", multiple=True)\n",
    "display(uploader)\n",
    "out = widgets.Output()\n",
    "display(out)\n",
    "\n",
    "def file_predict(filename, file, out):\n",
    "    \"\"\" A function for creating the prediction and printing the output.\"\"\"\n",
    "    image = tf.keras.utils.load_img(file, target_size=(150, 150))\n",
    "    image = tf.keras.utils.img_to_array(image)\n",
    "    image = rescale_layer(image)\n",
    "    image = np.expand_dims(image, axis=0)\n",
    "    \n",
    "    prediction = model.predict(image, verbose=0)[0][0]\n",
    "    \n",
    "    with out:\n",
    "        if prediction > 0.5:\n",
    "            print(filename + \" is a human\")\n",
    "        else:\n",
    "            print(filename + \" is a horse\")\n",
    "\n",
    "\n",
    "def on_upload_change(change):\n",
    "    \"\"\" A function for geting files from the widget and running the prediction.\"\"\"\n",
    "    # Get the newly uploaded file(s)\n",
    "    \n",
    "    items = change.new\n",
    "    for item in items: # Loop if there is more than one file uploaded  \n",
    "        file_jpgdata = BytesIO(item.content)\n",
    "        file_predict(item.name, file_jpgdata, out)\n",
    "\n",
    "# Run the interactive widget\n",
    "# Note: it may take a bit after you select the image to upload and process before you see the output.\n",
    "uploader.observe(on_upload_change, names='value')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-8EHQyWGDvWz"
   },
   "source": [
    "### Visualizing Intermediate Representations\n",
    "\n",
    "You can also look again at the intermediate representations. You will notice that the output at the last convolution layer is even more abstract because it contains fewer pixels than before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-5tES8rXFjux"
   },
   "outputs": [],
   "source": [
    "# Define a new Model that will take an image as input, and will output\n",
    "# intermediate representations for all layers in the previous model after\n",
    "# the first.\n",
    "successive_outputs = [layer.output for layer in model.layers[1:]]\n",
    "visualization_model = tf.keras.models.Model(inputs = model.inputs, outputs = successive_outputs)\n",
    "\n",
    "# Prepare a random input image from the training set.\n",
    "horse_img_files = [os.path.join(train_horse_dir, f) for f in train_horse_names]\n",
    "human_img_files = [os.path.join(train_human_dir, f) for f in train_human_names]\n",
    "img_path = random.choice(horse_img_files + human_img_files)\n",
    "img = tf.keras.utils.load_img(img_path, target_size=(150, 150))  # this is a PIL image\n",
    "x = tf.keras.utils.img_to_array(img)  # Numpy array with shape (150, 150, 3)\n",
    "x = x.reshape((1,) + x.shape)  # Numpy array with shape (1, 150, 150, 3)\n",
    "\n",
    "# Scale by 1/255\n",
    "# Since you are not using a tf.data.Dataset you apply the normalization via a simple numpy vectorization\n",
    "x /= 255\n",
    "\n",
    "# Run the image through the network, thus obtaining all\n",
    "# intermediate representations for this image.\n",
    "successive_feature_maps = visualization_model.predict(x, verbose=False)\n",
    "\n",
    "# These are the names of the layers, so you can have them as part of the plot\n",
    "layer_names = [layer.name for layer in model.layers[1:]]\n",
    "\n",
    "# Display the representations\n",
    "for layer_name, feature_map in zip(layer_names, successive_feature_maps):\n",
    "  if len(feature_map.shape) == 4:\n",
    "\n",
    "    # Just do this for the conv / maxpool layers, not the fully-connected layers\n",
    "    n_features = feature_map.shape[-1]  # number of features in feature map\n",
    "\n",
    "    # The feature map has shape (1, size, size, n_features)\n",
    "    size = feature_map.shape[1]\n",
    "\n",
    "    # Tile the images in this matrix\n",
    "    display_grid = np.zeros((size, size * n_features))\n",
    "    for i in range(n_features):\n",
    "      x = feature_map[0, :, :, i]\n",
    "      x -= x.mean()\n",
    "      x /= x.std()\n",
    "      x *= 64\n",
    "      x += 128\n",
    "      x = np.clip(x, 0, 255).astype('uint8')\n",
    "\n",
    "      # Tile each filter into this big horizontal grid\n",
    "      display_grid[:, i * size : (i + 1) * size] = x\n",
    "\n",
    "    # Display the grid\n",
    "    scale = 20. / n_features\n",
    "    plt.figure(figsize=(scale * n_features, scale))\n",
    "    plt.title(layer_name)\n",
    "    plt.grid(False)\n",
    "    plt.imshow(display_grid, aspect='auto', cmap='viridis')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tFnBvcIrXWW2"
   },
   "source": [
    "## Wrap Up\n",
    "\n",
    "In this lab, you saw how compacting images affected your previous model. This is one technique to keep in mind especially when you are still in the exploratory phase of your own projects. You can see if a smaller model behaves just as well as a large model so you can have faster training. You also saw how easy it is to customize your images for this adjustment in size by simply changing a parameter in the `image_dataset_from_directory` method."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0rc1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
